1. Field
One or more embodiments of the present invention relate to image processing, and more particularly, to an image analysis method and apparatus for moving image segmentation.
2. Description of the Related Art
Research on segmentation associated with digital video signal processing has steadily attracted attention. The field of segmentation has become an essential part of various multimedia applications and new techniques are being applied for effective analysis and representation.
Video signal segmentation has applications in various fields and the segmentation result can be effectively used in display quality processing, video encoding, and object recognition. For example, in the field of display quality processing, if a foreground and a background in a video can be segmented from each other, a three-dimensional effect/realness can be improved by separately processing the foreground and the background. The foreground is important and may be in the center portion of a video screen. The foreground may include objects in good focus or near a camera, e.g., a key person or object in the screen. The background is the remaining part of the screen excluding the foreground and may be of lesser importance or no interest, such as a background image, a mountain, a tree, or a building.
In the field of video encoding, video encoding compression efficiency can be improved using video segmentation like in moving picture experts group (MPEG)-4, for example. To this end, an analog video signal input through an imaging medium such as a video camera may be converted into a digital video signal, segmentation may be performed to distinguish a foreground object area from a background area, and separate compression encodings may be performed on the background area and the foreground object area based on the segmentation result.
In the field of object recognition, in order to recognize a main object in a video, a foreground area in which the main object exists should be first detected from the video.
As mentioned above, video segmentation is required as precedent technology for various application fields and thus various precedent techniques have been disclosed. The following are representatives of precedent studies in the field of video segmentation.
First, “Non-Parametric Model for Background Subtraction” by Ahmed Elgammal, et al. (Lecture Notes in Computer Science, Volume 1843, ECCV 2000, pp. 751-767) describes a segmentation method using background subtraction. However, although background subtraction may be fast and simple, it can typically only be applied to a video taken by a still camera, i.e., only when a camera is in a still state.
A patent version of the literature “Representing Moving Images with Layers” by J. Y. A. Wang, et al. (IEEE Trans. Image Processing, vol. 3, pp. 625-638, September 1994), i.e., U.S. Pat. No. 5,557,684 entitled “System for Encoding Image Data into Multiple Layers Representing Regions of Coherent Motion and Associated Motion Parameters” describes a segmentation method using 2-dimensional (2D) motion estimation in which 2D motion is estimated and segmentation is performed based on the motion estimation result. However, motion estimation and segmentation should to be performed repetitively because 2D motion estimation may not be accurate until segmentation is performed. Moreover, 2D motion estimation may be applicable only on the assumption that an object in a video is piecewise planar.
“Motion Segmentation and Outlier Detection” by P. H. S. Torr (Ph.D. thesis, Univ. of Oxford, 1995) describes a segmentation method that first estimates 3D motion of main feature points such as corner points and segment those feature points into a foreground and a background based on their 3D motions. This segmentation method is a relatively accurate technique compared to those based on 2D motion analysis, but it may have low processing speed due to the large amount of computation required.
“Bi-Layer Segmentation of Binocular Stereo Video” by V. Kolmogorov, et al. (Proc. IEEE Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 407-414, 2005) describes a segmentation method based on the fact that a distance between each pixel and a camera provides importation information for segmentation. However, the disclosed segmentation method may not be applicable to general video, i.e., a video taken by a mono camera, since it involves calculating a depth from two videos taken by a stereo camera.
As mentioned above, much research on video segmentation has been conducted over the past years and extensive precedent studies are being carried out on video segmentation. However, a segmentation method for effectively performing segmentation according to the features of various videos has not yet been suggested and conventional segmentation techniques have strong points and weak points in terms of the range of video to which they can be applied, accuracy, and processing speed.